An Accelerated First-Order Method for Non-convex Optimization on Manifolds

Abstract We describe the first gradient methods on Riemannian manifolds to achieve accelerated rates in non-convex case. Under Lipschitz assumptions and Hessian of cost function, these find approximate first-order critical points faster than regular descent. A randomized version also finds second-order points. Both algorithms their analyses build extensively existing work Euclidean The basic operation consists running descent method (appropriately safe-guarded against non-convexity) current tangent space, then moving back manifold repeating. This requires lifting function from which can be done for example through exponential map. For this approach succeed, lifted (called pullback) must retain certain properties. As a contribution independent interest, we prove precise claims that effect, with explicit constants. Those are affected by curvature manifold, turn affects worst-case complexity bounds our optimization algorithms.